Data processing system and computer program product for support of system memory addresses with holes

ABSTRACT

A method, computer program product, and a data processing system for supporting memory addresses with holes is provided. A first physical address range allocated for system memory for an operating system run by a processor configured to support logical partitioning is virtualized to produce a first logical address range. A second physical address range allocated for system memory for the operating system is virtualized to produce a second logical address range. The first physical address range and the second physical address range are non-contiguous. Virtualization of the first and second physical address ranges is had such that the first logical address range and the second logical address range are contiguous. A memory mapped input/output physical address range that is intermediate the first physical address range and the second physical address range is virtualized to produce a third logical address range. A lowermost logical address of the third logical address range exceeds a respective upper most logical address of the first and second logical address ranges.

BACKGROUND OF THE INVENTION

1. Technical Field

The present invention relates generally to an improved data processingsystem and in particular to a method and data processing system forimproved operating system operation. Still more particularly, thepresent invention provides a method and data processing system forproviding virtualization of memory-mapped input/output physicaladdresses to provide a contiguous system memory address space.

2. Description of Related Art

In many data processing systems, input/output cache inhibited addressesare mapped into an address range of the system address space. Often, theinput/output cache inhibited addresses have physical addressesintermediate multiple physical address ranges of the system memoryallocated for operating system usage. In such an instance, theinput/output cache inhibited addresses present a memory “hole” ofphysical addresses that reside in between separate blocks of memoryaddressable by the operating system.

An operating system may fail to support system memory addresses withholes. For example, in some configurations of two memory arrays, theLinux operating system may fail to handle memory addresses with holes.In such a situation, the operating system kernel may be modified toaccommodate the memory hole. However, such a solution is time consumingand may jeopardize some server program functionality, availability, andthe ability of the data processing system to fully support an operatingsystem version with such deficiencies.

Accordingly, it would be advantageous to provide a data processingsystem that supports system memory addresses with holes. It would befurther advantageous to provide a data processing system that supportssystem memory addresses with holes without requiring modification of theoperating system kernel.

SUMMARY OF THE INVENTION

The present invention provides a method, computer program product, and adata processing system for supporting memory addresses with holes. Afirst physical address range allocated for system memory for anoperating system run by a processor configured to support logicalpartitioning is virtualized to produce a first logical address range. Asecond physical address range allocated for system memory for theoperating system is virtualized to produce a second logical addressrange. The first physical address range and the second physical addressrange are non-contiguous. Virtualization of the first and secondphysical address ranges is had such that the first logical address rangeand the second logical address range are contiguous. A memory mappedinput/output physical address range that is intermediate the firstphysical address range and the second physical address range isvirtualized to produce a third logical cache inhibited address range. Alowermost logical address of the third logical address range exceeds arespective upper most logical address of the first and second logicaladdress ranges.

BRIEF DESCRIPTION OF THE DRAWINGS

The novel features believed characteristic of the invention are setforth in the appended claims. The invention itself, however, as well asa preferred mode of use, further objectives and advantages thereof, willbest be understood by reference to the following detailed description ofan illustrative embodiment when read in conjunction with theaccompanying drawings, wherein:

FIG. 1 is a block diagram of a data processing system that provides forsupport of system memory addresses with holes depicted in accordancewith a preferred embodiment of the present invention;

FIG. 2 is a diagrammatic illustration of a memory device having anon-contiguous physical memory address space that may be virtualized forsupport of memory addresses with a hole in accordance with a preferredembodiment of the present invention;

FIG. 3 is a diagrammatic illustration of a software and hardwareconfiguration for implementing logical partition virtualization of amemory device for supporting memory addresses with a memory hole inaccordance with a preferred embodiment of the present invention;

FIG. 4 is a diagrammatic illustration of a memory device that provides acontiguous logical system memory address space in accordance with apreferred embodiment of the present invention; and

FIG. 5 is a flowchart of processing performed by a data processingsystem for configuring a memory device for operating system support ofmemory addresses with holes in accordance with a preferred embodiment ofthe present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

Referring to FIG. 1, a block diagram of a data processing system thatprovides for support of system memory addresses with holes is depictedin accordance with a preferred embodiment of the present invention. Dataprocessing system 100 may be a symmetric multiprocessor (SMP) systemincluding a plurality of processors 102 and 104 connected to system bus106. Preferably, processors 102 and 104 are adapted to run under alogical partitioning environment. For example, processors 102 and 104may be implemented as respective 970 PowerPC processors manufactured byInternational Business Machines Corporation of Armonk, N.Y., or asimilarly functional processor device. Alternatively, a single processorsystem may be employed. Also connected to system bus 106 is memorycontroller/cache 108, which provides an interface to local memory 109.I/O bus bridge 110 is connected to system bus 106 and provides aninterface to I/O bus 112. Memory controller/cache 108 and I/O bus bridge110 may be integrated as depicted.

Peripheral component interconnect (PCI) bus bridge 114 connected to I/Obus 112 provides an interface to PCI local bus 116. A number of modemsmay be connected to PCI local bus 116. Typical PCI bus implementationswill support four PCI expansion slots or add-in connectors.Communications links to clients data processing systems may be providedthrough modem 118 and network adapter 120 connected to PCI local bus 116through add-in connectors. Additionally, a system firmware 115 may beconnected to local bus 116.

Additional PCI bus bridges 122 and 124 provide interfaces for additionalPCI local buses 126 and 128, from which additional modems or networkadapters may be supported. In this manner, data processing system 100allows connections to multiple network computers. A memory-mappedgraphics adapter 130 and hard disk 132 may also be connected to I/O bus112 as depicted, either directly or indirectly.

Those of ordinary skill in the art will appreciate that the hardwaredepicted in FIG. 1 may vary. For example, other peripheral devices, suchas optical disk drives and the like, also may be used in addition to orin place of the hardware depicted. The depicted example is not meant toimply architectural limitations with respect to the present invention.

The data processing system depicted in FIG. 1 may be, for example, anIBM JS20 blade eServer pSeries system, a product of InternationalBusiness Machines Corporation in Armonk, N.Y., running the AdvancedInteractive Executive (AIX) operating system or LINUX operating system.

FIG. 2 is a diagrammatic illustration of a memory device, such as localmemory 109 of data processing system 100 shown in FIG. 1, having anon-contiguous physical memory address space that may be virtualized forsupport of memory addresses with a hole in accordance with a preferredembodiment of the present invention. In the illustrative example, memory200 provides 8 Gigabyte (GB), or 8192 MB, of system memory. The systemmemory is divided into two non-contiguous physical memory ranges.Particularly, memory 200 includes a first physical range 210 and asecond physical range 211 separated by a memory mapped input/output(MMIO) physical range 220 that presents an address hole disposedintermediate physical ranges 210 and 211. In the illustrative example,physical range 210 comprises a 2048 megabyte (MB) range of system memoryaddressable by the operating system, and physical range 211 comprises a6144 MB range of system memory addressable by the operating system.Physical range 210 comprises a physical address range from hexadecimal0x000000000 to 0x07FFFFFFF, and physical range 211 comprises a physicaladdress range from hexadecimal 0x100000000 to 0x15FFFFFFF. Cacheinhibited addresses are mapped into physical range 220 having an addressrange intermediate physical ranges 210 and 211. Thus, memory 200includes MMIO physical range 220 that presents a discontinuity in thephysical addresses of the system memory provided by physical ranges 210and 211. MMIO physical range 220 comprises 2048 MB of memory having aphysical memory address range from hexadecimal 0x080000000 to0xFFFFFFFF.

FIG. 3 is a diagrammatic illustration of a software and hardwareconfiguration for implementing logical partition virtualization of amemory device for supporting memory addresses with a memory hole inaccordance with a preferred embodiment of the present invention.Operating system 304 may be, for example, an implementation of the Linuxoperating system, another variant of the Unix operating system such asthe Advanced Interactive executive operating system (AIX), or the like.Operating system 304 runs applications 302. Logical partitioning (LPAR)management 306 is a software system layer that facilitates themanagement of LPAR resources. LPAR management 306 may comprise computerexecutable instructions maintained, for example, as a computer programproduct stored on a hard disk or system firmware, such as firmware 115of data processing system 100 shown in FIG. 1. LPAR management 306 maybe implemented as, for example, the Hypervisor system, a product ofInternational Business Machines Corporation in Armonk, N.Y. LPARmanagement 306 may additionally include computer executable logicallocated on a system memory device such as local memory 109 of dataprocessing system 100 shown in FIG. 1. For example, LPAR management 306may allocate a portion of physical ranges 210 or 211 for alogical-to-physical address mapping table, LPAR program and datastorage, or the like. Additionally, LPAR 306 may allocate a page tablefor enabling operating system 304 virtual addressing support. Addressspace of physical ranges 210 or 211 consumed by the mapping table, pagetable, LPAR program and data storage is unavailable for access by theoperating system.

LPAR management 306 manages O/S 304 access to system hardware such asprocessors 308 and memory 309 and input and output (I/O) devices 310.Processors 308 are representative of processors 102 and 104 of dataprocessing system 100 and memory 309 is representative of local memory109 of data processing system 100 shown in FIG. 1. Processors 308support logical partitioning and are configured to run under a logicalpartitioning environment in accordance with a preferred implementationof the invention.

FIG. 4 is a diagrammatic illustration of a memory device, such as memory309 shown in FIG. 3, that provides a contiguous logical address space tothe operating system of data processing system 100 of FIG. 1 inaccordance with a preferred embodiment of the present invention. Thephysical addresses of MMIO physical range 220 are virtualized intological addresses above any logical address required for mapping systemmemory. In the illustrative example, MMIO physical range 220 having aphysical address range of 0x080000000 to 0xFFFFFFFF is virtualized intological MMIO range 420 having logical addresses from 0x10080000000 to0x100FFFFFFF. Additionally, O/S 304 will be presented with system memoryhaving logical addresses 0 to the size of the physical memory (less anyoverhead required for LPAR management 306).

In an exemplary implementation, system memory is accessed by logicalrange 410 having logical addresses from 0x00000000000 to 0x001F5FFFFFFis presented to O/S 304 and LPAR 306 program, data storage, andlogical-to-physical mapping table consumes 32 MB of system memory. LPAR306 utilizes 128 MB of system memory for a page table used by processors308 for virtual address support. Thus, in the illustrative example, thesystem memory is addressed by logical range 410 that comprises logicaladdresses of 0x00000000000 to 0x001F5FFFFFF. MMIO physical range 220 isaddressed by logical MMIO range 420 that comprises logical addressesfrom 0x10080000000 to 0x100FFFFFFFF.

LPAR management 306 converts the logical addresses into correspondingphysical addresses for access to the physical memory. For example, theconversion may be performed by a table-lookup for logical range 410 anda range check for logical MMIO range 420. Logical memory start and endaddresses and corresponding physical memory start and end addresses thatmay be defined by a mapping table for memory address virtualization inaccordance with a preferred embodiment of the present invention aresummarized in Table A below. As shown by Table A, the lower most, orstart, address of logical MMIO range 420 is greater than the upper most,or end, address of logical range 410. TABLE A Physical Logical MemoryLogical memory Physical memory memory start-address end-addressstart-address end-address 0x00000000000 0x00077FFFFFF 0x0080000000x07FFFFFFF 0x00078000000 0x001EFFFFFFF 0x100000000 0x277FFFFFF0x001F0000000 0x001F5FFFFFF 0x002000000 0x007FFFFF Physical Logical MMIOLogical MMIO Physical MMIO MMIO start-address end-address start-addressend-address 0x10080000000 0x100FFFFFFFF 0x080000000 0x0FFFFFFFF

FIG. 5 is a flowchart of a routine performed by a data processingsystem, such as data processing system 100 of FIG. 1, for configuring amemory device for operating system support of memory addresses withholes in accordance with a preferred embodiment of the presentinvention. The routine begins (step 502), for example during a systemboot, and the data processing system processor(s) is configured in alogical partitioning environment (step 504). A first physical memoryarray or range of physical memory addresses, such as physical range 210,is virtualized into a corresponding first logical address range or array(step 506). A second memory array, such as physical range 211, that isnon-contiguous with the first memory array is then virtualized (step508). Virtualization of the second physical range is performed such thatthe virtual address range corresponding to the second memory array iscontiguous with the logical address range produced from virtualizationof the first memory array. Thus, the logical address ranges producedfrom virtualization of the first and second physical ranges result in acontiguous logical address range.

The MMIO physical address range is then virtualized into a third logicaladdress range (step 510). The virtualization of the MMIO physicaladdress range is performed such that the lower most logical address ofthe logical address range produced from virtualization of the MMIOphysical address range is greater than the upper most logical address ofthe logical address ranges produced from virtualization of physicalranges 210 and 211. The mapping table is then loaded into the systemmemory device for providing logical-to-physical address translation(step 512) and the routine then exits (step 514). Accordingly, theoperating system is advantageously presented with a single, contiguouslogical address space for access to system memory such as logical range410 shown in FIG. 4.

Thus, the present invention provides a data processing system thatsupports system memory addresses with holes. Non-contiguous physicaladdress ranges of system memory are virtualized into a contiguouslogical address space. A memory mapped input/output physical range isvirtualized into a logical range of memory having a lower most addressthat is above the upper most address of the logical range of the systemmemory. Advantageously, no modification of the data processing systemO/S kernel is required for supporting system memory addresses withholes.

It is important to note that while the present invention has beendescribed in the context of a fully functioning data processing system,those of ordinary skill in the art will appreciate that the processes ofthe present invention are capable of being distributed in the form of acomputer readable medium of instructions and a variety of forms and thatthe present invention applies equally regardless of the particular typeof signal bearing media actually used to carry out the distribution.Examples of computer readable media include recordable-type media, suchas a floppy disk, a hard disk drive, a RAM, CD-ROMs, DVD-ROMS, andtransmission-type media, such as digital and analog communicationslinks, wired or wireless communications links using transmission forms,such as, for example, radio frequency and light wave transmissions. Thecomputer readable media may take the form of coded formats that aredecoded for actual use in a particular data processing system.

The description of the present invention has been presented for purposesof illustration and description, and is not intended to be exhaustive orlimited to the invention in the form disclosed. Many modifications andvariations will be apparent to those of ordinary skill in the art. Theembodiment was chosen and described in order to best explain theprinciples of the invention, the practical application, and to enableothers of ordinary skill in the art to understand the invention forvarious embodiments with various modifications as are suited to theparticular use contemplated.

1. A method of supporting memory addresses with holes, the methodcomprising the computer implemented steps of: virtualizing a firstphysical address range allocated for system memory for an operatingsystem run by a processor configured to support logical partitioning toproduce a first logical address range; virtualizing a second physicaladdress range allocated for system memory for the operating system toproduce a second logical address range, wherein the first physicaladdress range and the second physical address range are non-contiguousand the first logical address range and the second logical address rangeare contiguous; and virtualizing a memory mapped input/output physicaladdress range that is intermediate the first physical address range andthe second physical address range to produce a third logical addressrange, wherein a lowermost logical address of the third logical addressrange exceeds a respective upper most logical address of the first andsecond logical address ranges.
 2. The method of claim 1, wherein thesteps of virtualizing the first physical address range, the secondphysical address range, and the memory mapped input/output physicaladdress range comprises maintaining a mapping table that definesphysical addresses and corresponding logical addresses.
 3. The method ofclaim 2, wherein maintaining the mapping table further comprisesmaintaining the mapping table in a physical address space allocated toone of the first and second physical address ranges, and wherein thephysical address space is unavailable to an operating system accessingthe first and second physical address ranges.
 4. The method of claim 1,wherein the third logical address range is non-contiguous with the firstlogical address range and the second logical address range.
 5. Themethod of claim 1, further comprising: allocating a portion of at leastone of the first physical address range and the second physical addressrange for a logical partitioning management software layer.
 6. Themethod of claim 1, wherein the memory mapped input/output physicaladdress range is allocated for cache inhibited addresses.
 7. A computerprogram product in a computer readable medium for virtualizingnon-contiguous physical memory ranges into a contiguous logical addressrange, the computer program product comprising: first instructions forstoring logical-to-physical memory address translations for first andsecond non-contiguous physical address ranges of a memory deviceallocated for system memory and a third physical address rangecomprising a memory mapped input/output physical address range that isintermediate the first and second physical address ranges, and wherein alower most logical address of the third physical address range providedby the logical-to-physical memory address translations is greater thanan upper most logical address of first and second logical address rangesprovided by the logical-to-physical memory address translationscorresponding to the first and second non-contiguous physical addressranges; and second instructions, responsive to execution of the firstinstructions, for converting a logical address into a correspondingphysical address.
 8. The computer program product of claim 7, whereinthe logical-to-physical memory translations are stored in a mappingtable that is unavailable to an operating system accessing the memorydevice.
 9. The computer program product of claim 8, wherein the mappingtable is maintained in at least one of the first and second physicaladdress ranges.
 10. The computer program product of claim 7, wherein thesecond instructions provide logical partitioning functionality.
 11. Thecomputer program product of claim 7, wherein the second instructions aremaintained in at least one of the first and second physical addressranges.
 12. The computer program product of claim 7, wherein the secondinstructions interface an operating system with input and output devicesof a data processing system.
 13. The computer program product of claim12, wherein the second instructions present a contiguous logical addressrange comprising the first and second logical address ranges to theoperating system.
 14. The computer program product of claim 8, whereinthe third physical address range is allocated for cache inhibited memorymapped input/output addresses.
 15. A data processing system forsupporting non-contiguous system memory arrays, comprising: a memorythat contains first and second non-contiguous physical memory arraysallocated for system memory having respective first and second physicaladdress ranges and a third physical memory array having a third physicaladdress range intermediate the first and second physical address ranges,a data set, and a set of instructions; and a processor configured tosupport logical partitioning, wherein the processor, responsive toexecution of the instructions, is presented with a contiguous logicaladdress range for accessing the first and second non-contiguous memoryarrays.
 16. The data processing system of claim 15, wherein the data setis a mapping table defining logical-to-physical memory addresstranslations.
 17. The data processing system of claim 15, wherein theset of instructions provides logical partitioning management.
 18. Thedata processing system of claim 15, wherein the data set is maintainedin the memory in at least one of the first and second physical addressranges.
 19. The data processing system of claim 15, wherein the set ofinstructions are maintained in the memory in at least one of the firstand second physical address ranges.
 20. The data processing system ofclaim 15, wherein a second logical address range is mapped to the thirdphysical address range, and a lower most logical address of the secondlogical address range is greater than an upper most logical address ofthe contiguous logical address range.